Deep Audio-visual Learning: A Survey

Abstract Audio-visual learning, aimed at exploiting the relationship between audio and visual modalities, has drawn considerable attention since deep learning started to be used successfully. Researchers tend leverage these two modalities improve performance of previously considered single-modality tasks or address new challenging problems. In this paper, we provide a comprehensive survey recent audio-visual development. We divide current into four different subfields: separation localization, correspondence generation, representation learning. State-of-the-art methods, as well remaining challenges each subfield, are further discussed. Finally, summarize commonly datasets challenges.